Skip to content

[Streams] Update data quality calculations#4566

Merged
mdbirnstiehl merged 11 commits intoelastic:mainfrom
mdbirnstiehl:streams-data-quality-updates
Jan 13, 2026
Merged

[Streams] Update data quality calculations#4566
mdbirnstiehl merged 11 commits intoelastic:mainfrom
mdbirnstiehl:streams-data-quality-updates

Conversation

@mdbirnstiehl
Copy link
Contributor

This PR closes #4345 and updates information on how Streams calculates data quality.

@github-actions
Copy link
Contributor

github-actions bot commented Jan 8, 2026

✅ Vale Linting Results

No issues found on modified lines!


The Vale linter checks documentation changes against the Elastic Docs style guide.

To use Vale locally or report issues, refer to Elastic style guide for Vale.

@github-actions
Copy link
Contributor

github-actions bot commented Jan 8, 2026

🔍 Preview links for changed docs

@mdbirnstiehl mdbirnstiehl marked this pull request as ready for review January 8, 2026 21:44
@mdbirnstiehl mdbirnstiehl requested a review from a team as a code owner January 8, 2026 21:44
Copy link
Contributor

@nastasha-solomon nastasha-solomon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some light suggestions. Take em or leave em!

Use the following components to monitor the health of your data and identify and fix issues:

## Failure store
* **Degraded documents:** Documents from the last backing index of the stream with the `ignored` property, usually because of malformed fields or exceeding the limit of total fields when `ignore_above` is set to `false`. This component shows:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, the ignore_above setting requires a number, not a boolean: https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/ignore-above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review @ElenaStoeva! Would you mind taking another look to make sure I got that update correct?

Copy link

@ElenaStoeva ElenaStoeva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating these docs! Everything lgtm apart from one inconsistency around the ignore_above setting.

Copy link

@ElenaStoeva ElenaStoeva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes, latest lgtm!

@mdbirnstiehl mdbirnstiehl requested a review from a team as a code owner January 13, 2026 23:02
@mdbirnstiehl mdbirnstiehl merged commit f0f1bf1 into elastic:main Jan 13, 2026
8 checks passed
@mdbirnstiehl mdbirnstiehl deleted the streams-data-quality-updates branch January 13, 2026 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Streams] Update docs for data quality

4 participants